Wire the kernel backend's two blocking cgo calls to the new cancellable
C-ABI entry points via a ctxWatcher that bridges a Go context onto a kernel
cancel token:

  - OpenSession -> kernel_session_open_cancellable
  - rows.go nextBatch -> kernel_result_stream_next_batch_cancellable

Previously each blocked in an uninterruptible cgo call: a slow warehouse
cold-start or a hung CloudFetch chunk ignored the caller's ctx deadline. The
watcher fires the token on ctx.Done(), which drops the in-flight kernel request
future (a real abort), and the call sites prefer the ctx error on cancellation
while preserving the kernel error as cause (matching the execute path and the
database/sql convention). A session-fatal error is evicted before the
ctx-cancelled return so a failure racing a cancel still evicts the conn. A
non-cancellable ctx (nil Done) yields a nil watcher -> NULL token -> the plain,
unchanged path, so there is zero watcher overhead on the common case.

Requires the kernel cancel-token symbols; bump KERNEL_REV to the merged kernel
revision before this builds in CI (it links a locally-staged archive today).

Tests: tagged ctxWatcher unit tests (fires on cancel/deadline, nil-safe on an
uncancellable ctx, clean teardown without fire) exercising the real cgo
ctx->token bridge in the build-and-test-kernel CI job. Default CGO_ENABLED=0
build unchanged.

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

…etch cancel

Address PR #409 review findings:
- OpenSession's cancelled-connect path now wraps BOTH the ctx error and the
  underlying kernel error (two %w), so errors.Is still matches the ctx error
  while the *KernelError stays reachable via errors.As. Mirrors the execute
  (operation.go) and next_batch (rows.go) cancelled paths, which already did
  this; the connect path was dropping the kernel diagnostics.
- Add TestKernelE2ECancelDuringFetch, which deterministically reaches
  rows.Next() after QueryContext succeeds and then observes cancellation during
  fetch — closing the coverage gap where the read-path cancel could pass without
  exercising kernel_result_stream_next_batch_cancellable.

Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

The previous TestKernelE2ECancelDuringFetch cancelled between batches, so
nextBatch's pre-fetch ctx.Err() guard short-circuited before ever calling
kernel_result_stream_next_batch_cancellable — it only proved the read path
honored ctx, not the in-flight token abort.

Rework it to cancel while a fetch is actually blocked: a watcher goroutine fires
the cancel only once a single rows.Next() has been blocked longer than any
buffered-row return could take (i.e. it is inside the network fetch). The run is
verified to reach the cancellable call by the "next_batch cancelled" wrapper the
C-call error path emits (the pre-fetch guard returns a bare context error);
attempts are retried a bounded number of times to absorb the timing window.

Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

…ions

Wire the four new kernel C-ABI symbols on the driver side, extending the
existing experimental-config surface:

  - checkABIVersion(): a sync.Once handshake at the top of OpenSession compares
    the linked library's kernel_abi_version() against the header's
    DATABRICKS_KERNEL_ABI_VERSION and refuses to connect on a mismatch, so a
    differently-built prebuilt .a can't be silently misread.
  - WithKernelClientCertificate(cert, key): mTLS client identity forwarded via
    the paired kernel_session_config_set_tls_client_certificate. Both PEM halves
    are required together — validateKernelConfig rejects an unpaired credential
    loudly so a lone key can't be silently dropped — and the key is never logged.
  - WithKernelCloudFetch(enabled): a tri-state *bool (nil keeps the kernel
    default on; set forwards kernel_session_config_set_cloudfetch_enabled).
    Distinct from the plain-bool WithCloudFetch, whose unset state can't be told
    from false.

All three are kernel-only and rejected loudly on the Thrift path (the connector
fails when KernelExperimental is non-nil). The reflective classification guard is
extended so a new experimental field can't ship unforwarded/unrejected.

CloseSession stays fire-and-forget: the C ABI now also offers
kernel_session_close_blocking, but adopting it would make close a blocking
network round-trip with no deadline honored, so swapping to it is grouped with
the cancellable-close follow-up.

Requires the kernel ABI-version / mTLS / CloudFetch symbols; bump KERNEL_REV to
the merged kernel revision before this builds in CI (it links a locally-staged
archive today).

Tests: experimental-field classification guard + option wiring + Thrift
rejection + DeepCopy (untagged); TestSetKernelTLS mTLS case, TestABIVersionMatches,
and mTLS-pairing validation (tagged / config). Default CGO_ENABLED=0 build
unchanged.

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

The consolidated branch now carries the log-callback forwarding
(log_callback.go references kernel_set_log_callback / kernel_log_callback),
so KERNEL_REV must point at a kernel revision that exports those symbols.
Bump from the tier2 rev 02c0a434 (pre-log-callback) to 946c263, the head of
kernel PR #169 (mani/c-abi-log-callback), which defines them. The kernel-lib
build resolves this unmerged SHA via its PR-head-ref fallback, so the tagged
kernel-backend build links a header carrying the symbols.

Temporary: re-pin to the squash-merge SHA once kernel #169 lands.

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

Test coverage:
- Assert buildKernelConfig forwards TLSClientCertPEM / TLSClientKeyPEM /
  CloudFetchEnabled (previously only trusted-certs + hostname-skip were
  asserted, so a dropped forwarding line could ship green), plus an
  unset-CloudFetchEnabled-stays-nil case.
- Add a tagged, hermetic mid-connect cancel test: a black-hole TLS listener
  makes OpenSession block, and a short ctx deadline must fire the cancel token
  mid-connect and surface as "session_open cancelled" matching
  DeadlineExceeded. The prior connect-cancel e2e only hit the pre-connect
  guard; its docstring is corrected to say so.

Contract:
- Reject a bare WithCloudFetch(false) on the kernel path (steering to
  WithKernelCloudFetch(false)) instead of silently dropping it and leaving
  CloudFetch on; classify UseCloudFetch as rejected. When WithKernelCloudFetch
  is set it stays authoritative.
- doc.go: mid-fetch cancellation is now supported (not a limitation); refresh
  the "nothing silently ignored" list.

Operational:
- Forward kernel ERROR/WARN one severity lower by default (->Warn/Info, tagged
  with the native level) so kernel-internal retries don't inflate the driver's
  alert-keyed WARN/ERROR rate; DBSQL_KERNEL_DEBUG preserves native severity.

Cleanup:
- Extract cancelledErr() for the shared dual-%w cancelled-path wrap used by
  OpenSession, execute, and nextBatch.
- Create one ctxWatcher per result set (in newKernelRows, stopped in Close)
  instead of one per Arrow batch, matching the execute path's amortization.

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

…erage

Add a tagged, warehouse-free test that exercises the experimental kernel TLS
options end-to-end through the real cgo -> kernel -> rustls stack against a local
httptest TLS server:

- WithKernelClientCertificate: with a client cert the mTLS handshake completes
  (handler reached, peer cert observed); against a RequireAndVerifyClientCert
  server with no cert, the handshake is rejected before the handler runs.
- WithKernelTrustedCerts: a privately-signed server cert validates only when its
  CA is trusted; an untrusted CA fails the chain.
- WithKernelSkipHostnameVerify: a wrong-SAN server cert is rejected on hostname
  grounds unless the skip is set.

Certs are minted at runtime with crypto/x509 (PKCS#8 keys, the form the kernel's
tls-rustls Identity::from_pem accepts) — no committed fixtures, no external tools.
The kernel speaks SEA so the query itself fails at the app layer; the property
under test is that the handshake completes with the forwarded material, observed
via a per-connection reached/saw-cert probe. Each case cancels the connect the
instant the outcome is decided rather than riding the kernel's connect-retry
budget, so the file runs in a few seconds. Runs in the tagged build-and-test-kernel
job (no warehouse needed) — closing the mTLS/custom-CA gap that had only
byte-marshalling unit coverage before.

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

Follow-ups on the ctx-cancel backend, all behind the cgo && databricks_kernel
tag. No behavior change on the default CGO_ENABLED=0 Thrift build.

- OpenSession / nextBatch: emit a debug-gated klogCtx line when a connect or
  mid-fetch is interrupted by the caller's context, right before returning the
  cancelled error, so a ctx-driven cancel is visible in kernel debug logs
  (zero cost when kernel debug logging is off).
- cancel.go: reword the ctxWatcher doc to match kernel behavior — firing the
  token is a real abort on connect (request awaited inline) but a prompt
  stop-waiting on mid-fetch (the CloudFetch download runs on a detached kernel
  task and drains to its ~60s read-timeout); either way the caller's goroutine
  is freed at the deadline. Mirrors the reworded kernel header contract.
- abi.go: fix a stale comment — checkABIVersion runs abiVersions() once per
  process (sync.Once-cached), not on every connect.

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

Point KERNEL_REV at c19e5d1, the current head of kernel PR #169. The
prior pin (946c263) was an orphaned SHA from an earlier force-push of the
same PR branch. ABI version is unchanged (still 1 — the earlier bump was
reverted), and all driver-used C-ABI symbols are present in the header at
the new rev.

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

Forward a backend-neutral WithCloudFetch(false) to the kernel CloudFetch
toggle instead of rejecting it at connect. UseCloudFetch defaults to true,
so false is an unambiguous explicit disable — honoring it (equivalent to
WithKernelCloudFetch(false)) is friendlier than the previous hard error and
still satisfies the "nothing silently ignored" contract. An explicit
WithKernelCloudFetch stays authoritative when both are set.

This unblocks the driver-test Go integration suite, whose result-fetching
tests disable CloudFetch via the backend-neutral cfg.UseCloudFetch on the
SEA leg (TestLazyFetchEmptyResultSet).

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

…ath"

This reverts commit 15d276b.

Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>

Match the slimmed kernel PR (#169 → mTLS + ABI-version only): remove the
Go wiring for the C-ABI symbols dropped kernel-side, re-pin KERNEL_REV to
the updated kernel head (12919b3), and keep the two survivors.

Keep:
- WithKernelClientCertificate → set_tls_client_certificate (mTLS); the
  option, config plumbing, unit tests, and kernel_mtls_test.go hermetic
  handshake test all stay.
- checkABIVersion() / abiVersions() at connect (abi.go + abi_test.go +
  TestABIVersionMatches) — the drift handshake.

Remove (no merged consumer; kernel symbols gone):
- ctx-cancel bridge: cancel.go (ctxWatcher/cancelledErr), cancel_test.go,
  cancel_connect_test.go; revert OpenSession to plain kernel_session_open,
  rows.go nextBatch to plain kernel_result_stream_next_batch, and
  operation.go's execute-path wrap to its inline form.
- log-callback bridge: log_callback.go, logforward.go, logforward_test.go,
  and the TestLogCallback* cases; initKernelLogging reverts to the
  kernel_init_logging (stderr) path.
- CloudFetch toggle: WithKernelCloudFetch + the CloudFetchEnabled field and
  its validation/forwarding/classification (UseCloudFetch back to inert).
- close_blocking references (never wired; CloseSession stays fire-and-forget).

Reverted kernel_e2e_test.go to drop the three cancel-family e2e tests
(mid-fetch / query-cancel-during-execution / connect-honors-cancelled-ctx);
the pre-existing TestKernelE2ECancellation still covers query cancel.

Validated against the new kernel lib: untagged build/vet/tests green; tagged
build + vet + kernel-backend unit tests green; live e2e against a warehouse
green (Select1, DataTypes ×18, CloudFetch, Cancellation, M2M, TLSSkipVerify,
Interval, MetricViewMetadata — all PASS).

Co-authored-by: Isaac
Signed-off-by: Mani Kaustubh Mathur <mani.mathur@databricks.com>